<!--

    The contents of this file are subject to the license and copyright
    detailed in the LICENSE and NOTICE files at the root of the source
    tree and available online at

    http://www.dspace.org/license/

-->
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<html>
<head>
<!--
  Author:   Robert Tansley
  Version:  $Revision: 5844 $
  Date:     $Date: 2010-11-12 00:29:14 -0500 (Fri, 12 Nov 2010) $
-->
</head>
<body bgcolor="white">
<P>Provides an API for storing, retrieving and deleting streams of bits in
a transactionally safe fashion. The main class is
<a href="BitstreamStorageManager.html">BitstreamStorageManager</a>.</P>

<h2>Using the Bitstore API</h2>

<p>An example use of the Bitstore API is shown below:</P>
<pre>
    // Create or obtain a context object
    Context context;
    // Stream to store
    InputStream stream;
    
    try
    {
        // Store the stream
        int id = BitstreamStorageManager.store (context, stream);
        // Retrieve it
        InputStream retrieved = BitstreamStorageManager.retrieve(context, id);
        // Delete it
        BitstreamStorageManager.delete(context, id);

        // Complete the context object so changes are written
    }
    // Error with I/O operations
    catch (IOException ioe)
    {
       
    }
    // Database error
    catch (SQLException sqle)
    {
    }
</pre>

<h2>Storage mechanism</h2>

<p>The BitstreamStorageManager stores files in one or more <em>asset store</em>
directories.  These can be configured in <code>dspace.cfg</code>.  For
example:</P>

<PRE>assetstore.dir = /dspace/assetstore</PRE>

<P>The above example specifies a single asset store.</P>

<PRE>assetstore.dir = /dspace/assetstore_0
assetstore.dir.1 = /mnt/other_filesystem/assetstore_1</PRE>

<P>The above example specifies two asset stores.  <code>assetstore.dir</code>
specifies the asset store number 0 (zero); after that use 
<code>assetstore.dir.1</code>, <code>assetstore.dir.2</code> and so on.  The
particular asset store a bitstream is stored in is held in the database, so
don't move bitstreams between asset stores, and don't renumber them.</P>

<P>By default, newly created bitstreams are put in asset store 0 (i.e. the one specified by the
<code>assetstore.dir</code> property.)  To change this, for example when asset
store 0 is getting full, add a line to <code>dspace.cfg</code> like:</P>

<PRE>assetstore.incoming = 1</PRE>

<P>Then restart DSpace (Tomcat).  New bitstreams will be written to the asset
store specified by <code>assetstore.dir.1</code>, which is
<code>/mnt/other_filesystem/assetstore_1</code> in the above example.</P>
  

<H3>Moving an Asset Store</H3>

<P>You can move an asset store as a whole to a new location in the file system; stop DSpace
(Tomcat), move all of the contents to the new location, change the appropriate
line in <code>dspace.cfg</code>, and restart DSpace (Tomcat).</P>

<P>We will be providing administration tools for more sophisticated management
of these asset stores in the future.</P>

<P>When given a stream of bits to store, the BitstreamStorageManager
generates a unique key for the stream. The key takes the form of
a long sequence of digits, which is transformed into a file path.
The BitstreamStorageManager stores the contents of the stream in
this path, creating parent directories as necessary.
</p>


<h2>The Bitstore and Transactions</h2>

<p>
The bitstore is carefully engineered to prevent data loss, using
transactional flags in the database. Before a bitstream is 
actually stored, a metadata entry with the unique bitstream id is
committed to the database. If the storage operation fails or is
aborted, the deleted flag remains. The bitstore API then ensures
that the bitstream cannot be retrieved, and after an hour, the
bitstream is eligible for cleanup. The bitstream is accessible only
after all database operations have been successfully committed.
</p>

<p>
Similarly, bitstreams are deleted by simply setting the deleted
flag. If an deletion operation is rolled back, the bitstream is still
present in the asset store.
</p>

<h2>Cleaning up the Asset Store</h2>

<P>As noted above, sometimes files will be physically present in the
Asset Store even though they are marked deleted in the database.
You can use the command-line utility class
<code>org.dspace.storage.bitstore.Cleanup</code> (which is invoked via
<code>/dspace/bin/cleanup</code>)
to remove the bitstreams which are marked deleted from the Asset Store.
To prevent accidental deletion of bitstreams which are in the process
of being stored, cleanup only removes bitstreams which are more than
an hour old.</P>
</body>
</html>

